Conventionally, there is a moving image generation device for displaying a part of a panorama moving image on a display device and outputting a sound when displaying the panorama moving image.
However, the above-described moving image generation device outputs the same sound regardless of which part of the panorama moving image is displayed, and therefore does not easily give a viewer a sense of being at the site of the image when displaying the panorama moving image.
Accordingly, an object of the example embodiment is to provide a storage medium storing an information processing program, an information processing device, an information processing system, and an information processing method capable of, when displaying a panorama moving image, giving a viewer an enhanced sense of being at the site of the image.
In order to achieve the above object, the example embodiment may adopt, for example, the following structures. It is understood that for interpreting the recitations of the claims, the range thereof is to be interpreted only based on the recitations of the claims, and that in the case where the recitations of the claims are contradictory to the description of the specification, the recitations of the claims are given priority.
An exemplary configuration of a non-transitory computer-readable storage medium according to the exemplary embodiment stores an information processing program which is executable by a computer included in an information processing device for displaying an image on at least one first display device including a speaker. The information processing program allows the computer to execute reading and sequentially acquiring a panorama moving image from a panorama moving image storage unit storing a panorama moving image; setting a display range, of the acquired panorama moving image, which is to be displayed on the first display device in accordance with an operation made by a user; generating, based on the display range, a sound to be output in synchronization with display of the image on the first display device; performing control of sequentially displaying the display range of the acquired panorama moving image on the first display device; and performing control of outputting the generated sound from the speaker in synchronization with the display of the image on the first display device.
The “information processing device” described above may be formed of a device different from the first display device. Alternatively, in the case where the first display device has an information processing function, the “information processing device” may be formed of the first display device. In the former case, the “different device” may execute each process of the example embodiment, whereas the first display device may merely display an image generated by the “different device”. Alternatively, in the case where the first display device has an information processing function, each process of the example embodiment may be realized by a cooperation of the information processing function of the first display device and the information processing function of the “different device”. The “different device” may be formed of a plurality of information processing devices, and execution of each process of the example embodiment may be divided among the plurality of information processing devices. The “information processing device” may be a game device provided as an example in the example described later or a multi-purpose information processing device such as a general personal computer.
In an example, the first display device is a portable display device. In another example, the first display device is a non-portable display device. In these cases, the first display device is a display device connected to the information processing device in a wireless or wired manner, such as a terminal device 2 or a monitor 4 in the example described later. The first display device may be any device capable of displaying an image generated by the information processing device. For example, the first display device may be integrated with the information processing device (accommodated in the same housing with the information processing device).
The “panorama moving image” described above may have an angle of field which is equal to or greater larger than 180° in one of an up-down direction and a left-right direction. Further, the “panorama moving image” may have an angle of field of 360° in one of the directions. In the other direction, the “panorama moving image” may have an angle of field which is equal to or greater than that of the moving image to be displayed on the first display device. Further, the “panorama moving image” may have an angle of field which is equal to or greater than twice the angle of field of the moving image, equal to or greater than 120°, equal to or greater than 150°, or 180°.
The “panorama moving image storage unit” may store a moving image captured by a panorama moving image capturing function of the information processing device, or may store a moving image captured by another device having the moving image capturing function and transmitted thereto via a prescribed storage medium or a network.
According to the above, the sound to be output from the speaker is adjusted in accordance with the display range of the panorama moving image displayed on the first display device. Therefore, when the panorama moving image is displayed, an enhanced sense of being at the site of the image can be provided.
The information processing program may allow the computer to further execute acquiring an original sound for generating the sound to be output in synchronization with the display of the image on the first display device, the original sound being acquired from a sound storage unit storing at least one sound recorded at the time of capturing the panorama moving image or at the time of reproducing the panorama moving image. In this case, a volume of the acquired original sound may be adjusted based on the display range to generate the sound to be output in synchronization with the display of the image on the first display device.
According to the above, the sound to be output when the panorama moving image is displayed can be generated by use of the sound recorded at the time of capturing the panorama moving image or at the time of reproducing the panorama moving image.
The sound storage unit may store a position in the panorama moving image which is a sound source of the recorded sound, the position being stored in correspondence with each or the at least one sound. Based on the position of the sound source with respect to the display range of the panorama moving image, the volume of the sound having the position as the sound source may be adjusted.
According to the above, the localization at the time of outputting the sound recorded at the time of capturing the panorama moving image or at the time of reproducing the panorama moving image can be adjusted in accordance with the panorama moving image to be displayed.
In the case where the display range of the panorama moving image and the position of the sound source are relatively close to each other, the volume of the sound having the position as the sound source may be adjusted to be relatively large.
According to the above, the sound at the sound source within, or close to, the display range on the first display device can be output with priority. Therefore, when the panorama moving image is displayed, an enhanced sense of being at the site of the image can be provided.
In the case where the position of the sound source is within a prescribed range with respect to the display range of the panorama moving image, the volume of the sound having the position as the sound source may be set to a value equal to or greater than a prescribed value, and in the case where the position of the sound source is outside the prescribed range, the volume of the sound having the position as the sound source may be set to 0.
According to the above, the sound output can be controlled such that the sound at the sound source is turned ON or OFF in accordance with whether or not the sound source is in the prescribed display range with respect to the display range.
The sound storage unit may store sounds respectively recorded at a plurality of sound sources at the time of capturing the panorama moving image or at the time of reproducing the panorama moving image, and may store a position, in the panorama moving image, of each of the sound sources, the position being stored in correspondence with each of the sounds. The sound to be output in synchronization with the display of the image on the first display device may be selected based on the position of each of the sound sources with respect to the display range of the panorama moving image.
According to the above, the sound to be output can be selected by changing the display range of the panorama moving image.
In the case where the display range of the panorama moving image and the position of each of the sound sources are put into a prescribed relationship, the sound having the position as the sound source may be selected as the sound to be output in synchronization with the display of the image on the first display device.
According to the above, the sound to be output can be selected by moving the display range of the panorama moving image.
The volume of the sound having the position as the sound source may be adjusted in accordance with whether the display range of the panorama moving image encompasses the position of the sound source or not.
According to the above, only the sound at the sound source which is within the display range of the panorama moving image can be output.
In the case where the display range of the panorama moving image encompasses the position of the sound source, the volume of the sound having the position as the sound source may be adjusted to be relatively large.
According to the above, the sound at the sound source within the display range displayed on the first display device can be output with an increased volume. Therefore, when the panorama moving image is displayed, an enhanced sense of being at the site of the image can be provided.
As the position of the sound source in the panorama moving image, the sound storage unit may store an effective area in which the sound having the position as the sound source is an output target, the effective area being stored as being set in the panorama moving image and in correspondence with each sound source. The volume of the sound at the sound source for which the effective area is set may be adjusted in accordance with the size of the effective area encompassed in the display range of the panorama moving image.
According to the above, the sound output can be controlled in accordance with the ratio of the effective area of each sound source with respect to the display range of the panorama moving image.
The first display device may be capable of displaying the panorama moving image as being enlarged or reduced in accordance with an operation made by the user. The display range to be displayed on the first display device may be set as being enlarged or reduced in accordance with an operation of enlarging or reducing the panorama moving image. The volume of the sound may be adjusted in accordance with the enlarged or reduced display range.
According to the above, the sound output can be controlled in accordance with the enlargement or the reduction of the moving image displayed on the first display device.
The panorama moving image storage unit may store a panorama moving image of a real world. The sound storage unit may store sounds which are recorded at the time of capturing the panorama moving image by use of a plurality of directional microphones having different sound collection directions in the real world, and may store a position corresponding to each of the sound collection directions in the panorama moving image as the position of the sound source, the position being stored in correspondence with each of the sounds. The volume of the sound having the position as the sound source may be adjusted based on the position of the sound source with respect to the display range of the panorama moving image.
According to the above, the sound to be output when the panorama moving image is displayed can be generated by use of the sounds recorded by the plurality of directional microphones.
The panorama moving image storage unit may store a panorama moving image of a real world. The sound storage unit may store sounds which are recorded at the time of capturing the panorama moving image by use of microphones respectively provided at a plurality of sound collection positions in the real world, and may store each of the sound collection positions in the panorama moving image as the position of the sound source, the position being stored in correspondence with each of the sounds. The volume of the sound having the position as the sound source may be adjusted based on the position of the sound source with respect to the display range of the panorama moving image.
According to the above, the sound to be output when the panorama moving image is displayed can be generated by use of the sounds recorded by the microphones respectively provided at the plurality of sound collection positions.
The sound storage unit may store at least a sound recorded at the time of capturing the panorama moving image, and may store a position in the panorama moving image specified at the time of the recording as the position of the sound source, the position being stored in correspondence with each of the sounds. The volume of the sound having the position as the sound source may be adjusted based on the position of the sound source with respect to the display range of the panorama moving image.
According to the above, the sound to be output when the panorama moving image is displayed can be generated by use of the sound added at the time of reproducing the panorama moving image.
The information processing program may allow the computer to further execute locating the acquired panorama moving image in a virtual space. In this case, a line-of-sight direction toward the panorama moving image located in the virtual space may be changed to set the display range of the panorama moving image.
According to the above, the sound output control can be performed in accordance with the direction in which the panorama moving image is displayed.
The information processing program may allow the computer to further execute receiving an input from an input device associated to the first display device. In this case, the display range may be set based on the input received from the input device associated to the first display device.
According to the above, the display range of the panorama moving image can be changed, and the sound to be output from the first display device can also be changed, by use of the input device associated to the first display device.
The first display device may include a sensor for outputting data in accordance with a movement or an attitude of a main body of the first display device. The data output from the sensor may be received as an input from the input device. The display range may be set in accordance with the attitude of the first input device, which is calculated based on the data output from the sensor.
Typically, the “setting of the display range” described above is moving the range to be displayed on the first display device in accordance with the change of the attitude of the first display device, at least in the same direction as that of the change of the attitude. Alternatively, the “setting of the display range” is making the amount of movement larger as the amount of change of attitude of the first display device is larger. Still alternatively, the “setting of the display range” is matching the amount of change of the attitude of the first display device to the amount of change of the attitude of the virtual camera for generating the range of the moving image. The “setting of the display range” may be controlling the change of the current attitude from the reference attitude of the virtual camera in accordance with the change of the current attitude from the reference attitude of the first display device. Alternatively, the “setting of the display range” may be controlling the change of the current attitude from the immediately previous attitude of the virtual camera in accordance with the change of the current attitude from the immediately previous attitude of the first display device. The “attitude” may be set two-dimensionally or three-dimensionally.
According to the above, when the attitude of the first display device is changed, the display range of the panorama moving image in accordance with the change of the attitude is displayed and the sound in accordance with the display range is output. Therefore, an operation environment which makes the user feel as if he/she was looking at a space formed by the panorama moving image via the first display device can be provided.
The sensor may be at least one of a gyrosensor or an acceleration sensor for outputting the data in accordance with the movement or the attitude of the main body of the first display device. The display range may be set in accordance with the attitude of the first display device, which is calculated based on the data output from at least one of the gyrosensor and the acceleration sensor.
According to the above, the attitude of the main body of the first display device can be easily calculated by use of angular velocity data obtained from the gyrosensor or acceleration data obtained by the acceleration sensor.
A display range which is to be displayed on a non-portable second display device may be further set in the acquired panorama moving image. A sound to be output from the second display device in synchronization with display of an image on the second display device may be further generated independently from the sound generated based on the display range displayed on the first display device. Control of sequentially displaying, on the second display device, the display range of the acquired panorama moving image which is to be displayed on the second display device may be further performed. Control of outputting the sound to be output from the second display device, from the second display device in synchronization with the display of the image on the second display device may be further performed.
According to the above, the sound independent from the sound controlled in accordance with the display range to be displayed on the first display device can be output from the non-portable display device.
The panorama image which forms each frame of the panorama moving image may be read and sequentially acquired from the panorama moving image storage unit at a cycle of a predetermined time length for which reproduction is performed on the first display device. The display range of the acquired panorama moving image may be set. The sound to be output in synchronization with the display of the panorama image on the first display device may be generated based on the display range. Control of sequentially displaying the display range of the acquired panorama image on the first display device may be performed. Control of outputting the generated sound from the speaker in synchronization with the display of the panorama image on the first display device may be performed.
The example embodiment may be implemented in the form of an information processing device or an information processing system each including units for performing the operations described above, or in the form of an information processing method including the operations described above.
According to the example embodiment, the sound to be output from the speaker is adjusted in accordance with the display range of the panorama moving image to be displayed on the display device. Therefore, when the panorama moving image is displayed, an enhanced sense of being at the site of the image can be provided.
These and other objects, features, aspects and advantages of the example embodiment will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.